1. Field of the Invention
The present invention relates to an image data encoding method and device for compressing image data, an image data reconstructing method and device for reconstructing original image data from compressed image data, and a scene-changed frame detecting method and scene change recording/regenerating device for detecting a change of scenes in continuous image frames.
2. Description of the Prior Art
Conventionally, image data having enormously large amount of data, compared with numerical data, specifically gray-scale images and colored image data are stored and transmitted at a high speed with high quality only by efficiently encoding gradation values for each picture element.
An adaptive discrete cosine transform (hereinafter referred to as ADCT for abbreviation) is used to efficiently compress image data. The ADCT is a high-performance encoding method based on a two-dimensional discrete cosine transform (hereinafter referred to as a two-dimensional DCT).
The two-dimensional DCT was published as a proposed recommendation presented by the JPEG (Joint Photographic Experts Group), that is, a joint organization of the CCITT and the ISO which are developing the international standardization of a high-performance encoding of colored static images (referred to is "International Standard in Encoding Multimedia, 1991 version, written by Hiroshi Yasuda, published by Maruzen).
According to the above described ADCT, an input multivalue image are divided into a plurality of blocks each comprising 8.times.8 picture elements, an image signal (multiple data) for 8.times.8 picture elements in each block is transformed through a two-dimensional DCT to 8.times.8 two-dimensional DCT coefficients distributed as spatial frequencies, the two-dimensional DCT coefficients are quantized by a visually adaptive threshold, and the quantized coefficients are encoded to variable-length codes according to a statistically obtained Huffman table.
FIG. 1 is a basic block diagram showing a conventional image data encoding device for encoding a multivalue image according to the ADCT.
With the image data encoding device shown in FIG. 1, a multivalue image is divided into a plurality of blocks each comprising 8.times.8 picture elements, and inputted in block units from a terminal 134a to a DCT unit 134c through a block buffer 134b. The DCT unit 134c orthogonally transforms through a two-dimensional DCT an input image signal for 8.times.8 picture elements into 8.times.8 two-dimensional DCT coefficients distributed as spatial frequencies, and outputs them to a quantizing unit 134d.
The quantizing unit 134d linearly quantizes each of the inputted 8.times.8 two-dimensional DCT coefficients using a corresponding quantization threshold 134e in a quantization matrix not shown in FIG. 1. As a result, a two-dimensional DCT coefficient smaller than the above described corresponding quantization threshold 134e is set to "0", while DC components (direct current coefficient) and a small amount of AC components (alternating current coefficient) indicate a value other than "0".
As described above, quantized coefficients linearly quantized by the quantizing unit 134d are applied to a variable-length encoding unit 134f.
The variable-length encoding unit 134f refers to a Huffman code table comprising Huffman tables provided for DC components and AC components separately, and encodes an input quantized coefficient string to variable-length codes. That is, for DC components, it refers to a one-dimensional Huffman table, and encodes the difference between the leading DC component and the previous DC component into variable-length codes. For AC components, it refers to a two-dimensional Huffman table, and encodes to variable-length codes the combination of a significant coefficient having a value other than "0" (index) and the run-length of an insignificant coefficient having the value "0" before the significant coefficient. Thus, the variable-length encoding unit 134f encodes DC and AC components into variable-length codes according to the above described Huffman code table 134g, and outputs the encoded data from a terminal 134h.
In encoding a multivalue image according to the ADCT, a quantized coefficient can be determined according to a two-dimensional DCT coefficient and a corresponding quantization threshold 134e in a quantization matrix. The process is performed in block units on all the blocks of one image (one frame). Thus, two-dimensional DCT coefficients for one image (one frame) can be quantized.
FIG. 2 is a block diagram showing a static image reconstructing device for reconstructing an original image from data encoded to variable-length codes. The static image reconstructing device reconstructs an original image from encoded data applied to a terminal 135a through a variable-length code decoding unit 135b, a quantizing unit 135c, and an inverse DCT unit in the order reverse to that for the encoding process shown in FIG. 1. The reconstructed image is sequentially written at corresponding addresses of an image memory 135e to complete the reconstruction of one screen of an image.
That is, to explain further in detail, the data encoded by the static image encoding device are applied in block units from the input terminal 135a to the variable-length code decoding unit 135b via predetermined transmission lines.
The variable-length code decoding unit 135b decodes quantized coefficients containing DC and AC components in each block according to a Huffman code table 135f used in the above described variable-length encoding process.
A dequantizing unit 135c decodes the quantized coefficients containing DC and AC components in each block to two-dimensional DCT coefficients using a quantization threshold 135g used in the above described variable-length encoding process.
The inverse DCT unit 135d performs an inverse two-dimensional DCT using the decoded two-dimensional DCT coefficients in each block, and reconstructs an image signal for each picture element in each block.
The reconstructed image signal for each picture element in each block is written to the image memory 135e in block units.
A writing of an image signal for each picture element in each block to the image memory 135e can be controlled by an intra-block picture element write control unit 135h. That is, the intra-block picture element write control unit 135h controls a block address generating unit 135i such that the block address generating unit 135i outputs an address of a corresponding block in which a picture element signal is to be written, thereby controlling the writing of a reconstructed image in each frame to the image memory 135e in block units.
FIG. 3 shows the configuration of the basic part of a conventional moving picture encoding device for encoding moving picture data by an inter-frame prediction encoding method based on the interrelation between the images in the present and previous frames.
Moving picture in each scene is applied to a difference image generating unit 136b through an input terminal 136a.
An image memory 136c stores standard images (reconstructed images used as images in previous frames).
The difference image generating unit 136b obtains the difference image between a standard image stored in the image memory 136c and the present image (an image in the present frame) applied to the above described input terminal 136a, and stores the difference image in a block buffer 136d. The difference image is obtained for each block containing 8.times.8 picture elements. Therefore, the block buffer 136d stores the difference image (block difference image) indicating the difference between the images in the present and previous frames, each comprising 8.times.8 picture elements, at corresponding positions.
A DCT unit 136e performs a two-dimensional DCT on a difference block image comprising 8.times.8 picture elements stored in the block buffer 136d, obtains 8.times.8 two-dimensional DCT coefficients, and outputs them to a quantizing unit 136f.
The quantizing unit 136f linearly quantizes inputted two-dimensional DCT coefficients by a visually adaptive threshold, and obtains 8.times.8 quantized coefficients. The 8.times.8 quantized coefficients are zigzag scanned and outputted to a variable-length encoding unit 136g in the order of DC components, AC components having a low spatial frequency, and AC components having a high spatial frequency.
The variable-length encoding unit 136g encodes the inputted 8.times.8 quantized coefficients to variable-length codes according to a Huffman code table 136h, and stores the obtained encoded data in a code buffer 136i.
The 8.times.8 quantized coefficients outputted from the quantizing unit 136f are applied to a dequantizing unit 136i, and the dequantizing unit 136i decodes the coefficients to 8.times.8 two-dimensional DCT coefficients.
An inverse DCT unit 136k performs an inverse two-dimensional DCT using the 8.times.8 two-dimensional DCT coefficients, and reconstructs the difference block image.
An adding unit 136m adds the reconstructed difference block image to the previous image in the corresponding block in the basic image stored in the image memory 136c, and rewrites the resultant image to the image memory 136c.
The encoded data of a difference block image for all blocks of the present image are temporarily stored in the code buffer 136i by performing a series of the operations on all blocks of images in new frames inputted from the input terminal 136a, and then outputted by the terminal 136n. Thus, image data in the present frame are not encoded as is, but the image indicating the difference between the previous and present frames is encoded, thus successfully reducing the number of codes to be processed.
The difference image reconstructed using the encoded data of the difference image is added to the image in the image memory 136c. Then, the image memory 136c stores the image identical to that reconstructed by the reconstructing unit based on the encoded data of the difference image and put in the present frame. The image is used as an image in the previous frame of a standard image when an image in the next frame is applied from the input terminal 136a.
Next, FIG. 4 shows the configuration of the basic part of a conventional moving picture reconstructing device for reconstructing an original image from the encoded data outputted by the above described moving picture encoding device.
When the encoded data of the difference image are sequentially applied in block units from an input terminal 137a, a variable-length code decoding unit 137b decodes them to quantized coefficients in block units according to a Huffman code table 137c similar to the Huffman code table 136h in the conventional moving picture encoding device.
A dequantizing unit 137d dequantizes a decoded quantized coefficient in each block according to a quantization matrix (similar to the quantization matrix used by the dequantizing unit 136i in the conventional moving picture encoding device shown in FIG. 3) not shown in FIG. 4.
An inverse DCT unit 137e performs an inverse two-dimensional DCT in block units using decoded two-dimensional DCT coefficients in each block, and reconstructs a difference image in each block.
An adding unit 137f adds the reconstructed difference image in each block to an image in the corresponding block in an image memory 137g, and rewrites the resultant image in the image memory.
The writing of a reconstructed image in the image memory 137g for each block can be controlled by an intra-block picture element write control unit 137h. That is, the intra-block picture element write control unit 137h controls a block address generating unit 137i such that the block address generating unit 137i outputs to the image memory 137g an address of a block in which a difference image should be stored in the image memory 137g, thereby successfully writing a reconstructed image of each frame in block units.
The image memory 137g stores a reconstructed image of each frame by performing a series of the above described processes on encoded data of difference images of all blocks forming an image in each frame.
In a television broadcasting station which normally processes a large amount of pictures, picture signals accumulated in a storage device are edited. The editing operation is performed manually by an editor by combining picture sources (images) and regenerating pictures to be broadcast. At this time, the head of a combined picture source (change of scenes) can be visually identified in regenerating pictures.
With a video tape recorder for use by a consumer, when a plurality of picture sources are recorded in a roll of video tape, the portion (frame) to be regenerated can be detected by visually determining the change of pictures (scene-changed frame or change of scenes) while regenerating pictures at a high speed.
Thus, with a conventional picture recording/regenerating device, a change of scenes can be extracted visually by a user while regenerating pictures at a high speed.
In the above described editing process at a television station and the detecting process performed at home to detect a desired portion to be regenerated, different scenes contained in a series of recorded moving picture data would be recognized in a short time if the change of scenes could be automatically extracted from the series of recorded moving picture data. In this case, compared with a conventional method of visually detecting a desired scene while regenerating pictures at a high speed, a timesaving operation could be performed and the contents of the whole scenes could be easily recognized. However, such convenient devices have not been developed at all.
If a change of scenes can be automatically extracted from a series of recorded moving picture data, various significant effects can be expected when it is used with an image recording/regenerating device. For example, a recording system for recording moving picture data responsively to motions in pictures can be produced by controlling the recording system according to the extracted information on changes of scenes, only the changes of scenes can be regenerated among a series of recorded moving picture data by recording the extracted information on changes of scenes together with picture information or encoded data, or only the changes of scenes can be recorded and regenerated according to extracted information on changes of scenes.
Since moving picture is composed of a series of static images, the moving picture can be encoded and reconstructed by an encoder and a decoder to be used for processing static images. An encoder for static images has the advantage over that for moving picture that it is based on a simpler method and has a smaller circuit. However, since the above described static image encoding method has aimed at reducing the amount of data to be accumulated and transmitted, other applications such as automatically extracting frames containing changes of scenes from recorded moving picture data have never been expected at all.